IN THE CLAIMS : 

Please amend Claims 2^ 3, 5 and 8, 



Please add Claims 14-15 as follows. 



1. (Amended) A m e thod for r e covering 3D sc e n e structur e and cam e ra motion ftgm 
imag e data obtain e d from a multi imag e s e qu e nc e , wher e in a r e f e r e nc e imdg e of th e 
s e qu e nc e is talc e n by a camera at a r e f e r e nc e p e rsp e ctiv e and on e oy^or e succ e ssiv e 
imag e s of th e s e qu e nc e ar e tok e n at on e or mor e succ e ssiv e diff e r e nt p e rsp e ctiv e s by 
translating and/or rotating th e camera, th e m e thod compnsing th e st e ps of: An image 
processing method for recovery of a scene structure md camera motion from successive 
image data, comprising: / 

(a) comparing a d e t^nn^insnmag e data shifts for e ach succ e ssiv e imag e 
with r e sp e ct to th e reference image tVsUceessive images, wherein the successive images 
are taken by translating or rotating4h^^amera with respect to the reference image : ^ 
shifts b e ing d e riv e d from th e cam e m translation and/or rotation from th e r e fer e nc e 
p e rsp e ctiv e to th e succ e ssiv e difiE e r e nt persp e ctiv e s; 

(b) determiningy constructing a shift data matrix that incorporat e s the 
image data shifts for the successive each image with respect to the reference image ; 

(c) construaing a shift data representation that incorporates the image data 
shifts for each image including a first data record calculating a ranlc 1 factorization from 
tho shift data matrix uAng SVD, with one of the ranlc 1 factors being a v e ctor 
corresponding to the 3D structure and a second data record corresponding th e oth e r rank 

1 factor being a/ e ctor corr e sponding to the siz e of th e camera motions; 
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(d) dividing the successive images into smoothing windows; 

(e) determining r e covering the direction of camera motioH md 3D 
structure from the first and second data records between the windoAvj^ veetef 
corr e sponding to th e 3D structur e by solving a Hn e ar e quation; and 

(f) r e cov e ring th e 3D structur e by solving a lyl e ar e quation using th e 
r e cov e r e d cam e ra motion. 

2. (Currently Amended) The method of claim 1, wheifein, step (e) includes: 

computing a fir s t projection matrix matrices ; 

recovering camera rotation data records v e ctors and direction of camera 
translation from the shift data representation matyx, and th e first projection matrices 
matrix; 

computing a s e cond proj e ^tionjrfrarix; and 

r e cov e ring th e dir e ction of coimMra translation using th e shift data matrix, 
th e r e f e r e nc e imag e , th e s e cond proj e ction m^fa>x and th e r e cov e r e d cam e ra rotation 
v e ctors . 

3. (Currently Amended) The method of cl^im 2. wherein step tf^ (e) fiirther includes 
recovering the 3D structure from the shift flata representation matrix, th e referenc e imag e, 
the recovered camera rotation data record/ veetefs and the recovered direction of 
translation data record v e ctors . 

4. (Original) The method of claim l,iurther including preliminary steps of: 

recovering the rotations of the camera between each successive image; and 
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warping all images in the sequence toward the reference image, wl)ile 
neglecting the translations. 

5. (Ciirrently Amended) The method of claim 1, wherein step (b) ozmiprises: 

computing H and A^^^ , where H is a (a^^ - 3)x A?^ matrix defined so that 

HH^ is the identity matrix and H annihilates the three vect9rs , , where the 
three vectors are computed fi'om the reference image as 
= ^/.r(^^(p)),^^ = {v/.r(')(p)},^, = ^I'V^^^ipW where 



/-^^H^.J^). '•^'H^^H ^^'H^.3^)are defined by [r(*\ rR r(^)]^ 



-xy ] 



V ^ y 



and A is a shift data matrix, that gives me diffi^ in intensities between each 
successive image and the reference image is a mfj -\)xNp matrix with entries AI^ 

,where All the change in (smoothed) intpsily^ith respect to the reference image, 
and with no smoothing A/^ = - , wh^e iV, is the number of images, is the 
number of pixels, and where /' denotes me /-th image, with /=0. 1 . . ., iV; - 1 , and where 
1 1 = /' (/7„ ) denotes the image intensity/at the n-th pixel position in/' , where is the 
reference image, where x mdy are the /mage coordinates of the pixel position and 

p= (x,y) and where A^// = C ^AH^ i^v^here C is a constant matrix with and where the 
notation {V} used to denote a vector with elements given by the 

6. (Original) The method of cl/im 1, wherein step (c) comprises: 
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computing a rank-1 factorization of - A^^^, » M^'^S^*^^ where M^*^, !^^are 
vectors corresponding to the motion and structure respectively. 

7. (Original) The method of claim 1, wherein step (c) comprises: 

computing a rank-3 factorization of - A^^ « ^ M^^^S^^^^ where 

^(«)^ vectors corresponding to the motion and stn^ture respectively; 

setting Z„"* as constant within each windo^, where Z is the depth from the 

camera to a 3D scene along the cameras optical axis; 

listing each of the pixels so that those jti the A:-th smoothing window have 
sequential indices nk, (nk+1), . . . (nk+i - 1); 

computing a first projection matqx tty computing a Np x Np projection 
matrix ?^ which is block diagonal with zdroVitr^"B^ween different smoothing 
windows, and which annihilates the vectors {V/ M {/^ / and {/^ } where {V/} is a vector 

containing the gradient of the intensity at each mxel, and Ijc and are the gradients of the 
intensity in the reference image in the x and y ^rections; 

recovering the three camera rotation vectors includes solving the 
following equations 

Pj^(h^S(^^ - W"^)= 0 for thL 3-vector w^'^ ; 
computing a second projection matrix includes computing a N^xN^ 
projection matrix P-[^^ , which is block diagonal with zero entries between different 
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smoothing windows and annihilates (//'■5W)-4'w('') where w^") is the vector recovered 
previously; 

recovering the directions of camera translations by solving/for the 
directions of translation t^^^ via 

?^'i'fi''^{l^}-'f^''^{l,}^ and; 

recovering Z„ via 

(h^S"^). -irJ'^l =rW(z.-'(fWp, -[tyi).V/.)where [i'-'J, 
represents f ^^^f ^''^ the x and y component of the translation direction, and where r^"^ are 



constants. 

8. (Currently Amended) The method of claim 1, v/herein step (d) comprises: 

setting Z„"* as constant withir/ ead^v^ndow, where Z is the depth from the 

camera to a 3D scene along the camera^s optical ]^is; 

listing each of the pixels so that tlfose^ the ^-th smoothing window have 
sequential indices n^, (nk+1), . . . (nk+i - 1). 

9. (Original) The method of claim 2, whereinithe step of computing a first projection 
matrix includes computing a Np x Np projection matrix which is block diagonal with 
zero entries between different smoothing windows, and which annihilates the vectors 
{V/ - p}, {/^ } and [ly } where {V/} is a vectc/r containing the gradient of the intensity at 

each pixel, and I^c and are the gradient^of the intensity in the reference image in the x 
and j; directions. 
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10. (Original) The niethod of claini 2, wherein the step of recovering caniera ro^^tuon 
vectors includes solving the following equation 

Pf^(H^s(*) - 0 for the 3-vector w. 

1 1 . (Original) The method of claim 9 wherein, the step of commuting a second 
projection matrix includes computing ^ N^xN^ projection jnatrix P-^ , which is block 

diagonal with zero entries between different smoothing windows and annihilates 
- Tw where w is the vector recovered previously. 

12. (Original) The method of claim 2 wherein, tnjTstepOT recovering the direction 
camera translation includes solving for the directio/i ol^tr^slation T via 

Pf(-7;{/J--7;fcK^.{p-^^})= 

13. (Original) The method of claim 3 wherein/ step (f) includes, recovering Z„ via 

-[Wwl =Z:itp„ i[fl) yi„ where [t[ represents t% the 
X and y component of the translation directioi 



14. (New) A computer-readable medium tnat includes instructions for recovering scene 
structure and camera motion direction from successive image data obtained from a multi- 
image sequence containing a reference iniage and one or more successive images, 
whereby said one or more successive images of the sequence are taken at one or more 
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successive different perspectives by translating and/or rotating the camera wherein s^a 
instructions, when executed by a processor, cause the processor to: / 

a) compare a reference image to successive images, wherein the 
successive images is taken by translating or rotating the camera; / 

b) determine the image data shift for the successive^ images with respect to 
the reference image; / 

c) construct a shift data representation that incorporates the image data 
shifts for each image including a first data record correspiOnding to the 3D structure and a 
second data record corresponding to the camera motion^ 

e) divide the successive images into wifndows; 

f) determine the direction of the camera motion and 3D structure fi"om the 
first and second data record between the windows. / 

1 5. (New) A computer system for recoverinjg sce¥e structure and camera motion 
direction fi-om image data obtained fi-om a multicim^ge sequence containing a reference 
image and one or more successive images, whereby said one or more successive images 
of the sequence are taken at one or more successive different perspectives by translating 
and/or rotating the camera, comprising: I 
a processor; / 

a video source whose output is aigitized into a pixel map by a digitizer, 
said output is sent in electronic form via a system bus for access by main memory; 
a display means; / 

a user interaction means fo/selecting items on the display means; 
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a storage device in communication with said processor, s^idstorage 
device stores program code for programming said processor to perform a method 
comprising the steps of: 

(a) accessing stored digitized video bv^e processor from main memory; 

(b) determining the inmge data shjft for the successive image with respect 
to the reference inmgel 

(c) constructing a shift Aa^ repJesentation that incorporates the image data 
shifts for each image including a fi^data pecord corresponding to the 3D structure and a 
second data record corresponding to the camera motion; 

(d) dividing tHe successive images into windows; 

(e) detmnining the direction of the camera motion and 3D structure from 
the first and second data records between the windows. 
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